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DETAILED ACTION 

This is a Non-Final Office Action responsive to RCE filed 05/12/2008; 
Amendments/Remarks filed 04/1 1/2008. 

Claims 1-11, and 13-69 are currently pending in this application. Claim 12 was 
previously cancelled. Claims 1-2, 4-6, 9-10, 13-15, 17-18, 21-25, 28-29, 31, 34-42, 44- 
49, 51-59, 61-66, and 68-69 were previously presented. Claims 3, 7-8, 11,16, 19-20, 
26-27, 30, 32-33, 43, 50, 60, and 67 were original presented. Claims 1 , 21 , 34, 44, 51 , 
61, and 68-69 being independent claims; Effective filing date is 03/30/2004, (Assignee: 
Ricoh). 

Continued Examination Under 37 CFR 1.114 

A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1 .1 7(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
05/12/2008 has been entered. 
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Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as 
set forth in section 1 02 of this title, if the differences between the subject matter sought to be 
patented and the prior art are such that the subject matter as a whole would have been obvious 
at the time the invention was made to a person having ordinary skill in the art to which said 
subject matter pertains. Patentability shall not be negatived by the manner in which the invention 
was made. 

Claims 1-11, 13-14, 17-30, 33-41, 43-49, and 51-58 are rejected under 35 
U.S.C. 1 03(a) as being as being unpatentable over Chiu et al US007051 271 B1 filed 
10/03/2000 (hereinafter Chiu), in view of Hull [Document image similarity and 
equivalence protection] Published 11/29/119 by " IJDARUS" Pages 37-42 (hereinafter 
Hull), and further view of Bozdaqi et al -US006647535B1 1 filed 08/18/1999 (hereinafter 
Bozdagi), 

Regarding independent claims 1, 

Chui teaches: 

a method in a computer system for creating a composite electronic 
representation including presentation material information, 

(See Chui at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. annotations 
made on the paper documents during the meeting can be extracted and used as 
indexes to the video). This interpretation is supported by Applicant's specification, which 
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is stated "creating a composite electronic representation comprising the determined 
information" at Pages 3 Para 15. ) 

the method comprising: scanning a paper document to generate an 
electronic presentation of the document with presentation material. 

(See the Abstract and Col. 2, Lines 15-25^Chui discloses this limitation in that a 
computer system for creating a composite electronic representation including 
presentation material information (i.e. annotations made on the paper documents during 
the meeting can be extracted and used as indexes to the video.) in addition Chui furher 
discloses the a scanned document is linking to a segment of a video wherein the 
scanned document is linked to a first video frame in the plurality of video frames 
responsive to the comparison step includes an identifier for comparing to the plurality of 
video frame identifiers; 

This interpretation is supported by Applicant's Specification, which states "key 
frame images obtained from video information captured during the presentation" at 
Pages 21 Para 57. ) 

extracting a visual feature from the electronic presentation of the 
document, the visual feature corresponding to a portion of the presentation 
material; 

(See Chui at the Abstract, teaches annotations made on the paper documents during 
the meeting can be extracted and used as indexes to the video. An orthonormal 
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transform, such as a Digital Cosine Transform (DCT) is used to compare scanned 
documents to video frames.) 

accessing recorded information including at least one of audio and 
visual information recorded during a presentation of the presentation 
material; 

(See Chui at Col. 2, Lines 15-25, discloses a method for linking a scanned document to 
a segment of a video is provided. Whereby the scanned document identifier is then 
compared to the plurality of video frame identifiers; the scanned document is linked to a 
first video frame in the plurality of video frames responsive to the comparison step. 

Also see Chui at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 300 for 
accessing a segment of video file 200 which references scanned document 101a.) 

storing the composite electronic representation for access by the 
user or anther user accessing the composite electronic document. 

(See Chui at Col. 2, Lines 15-25, discloses a method for linking a scanned document to 
a segment of a video is provided. Whereby the scanned document identifier is then 
compared to the plurality of video frame identifiers; the scanned document is linked to a 
first video frame in the plurality of video frames responsive to the comparison step. 

Also see Chui at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 300 for 
accessing a segment of video file 200 which references scanned document 101a. 
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FIG. S 
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Also see Chui at Fig. 5 and Col. 8 Line 5 discloses memory item 806 and 
persistent storage 808 stores a scanned digital document 101a and video file 200.) 



In addition Chui does not teach, but Hull teaches: 

comparing the visual feature to the recorded information to 
determine a portion of the recorded information corresponding to the 
visual feature, whereby at least a portion of the recorded information 
corresponds to a feature portion of the presentation material; 
(See Fig. 2-33 and Section 4.1 "Similarity detection" Pages 39-40-> Hull discloses this 
limitation in that features extracted from the CCITT fax-compressed representations 
Of two images are compared to determine their visual similarity and whether they are 
equivalent.) 
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Figure X Document image similarity and equivalence detection algorithm. 
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Accordingly, It would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify Chui method of linking a video to a scan document to 
include a means of comparing the visual feature to the recorded information to 
determine a portion of the recorded information corresponding to the visual feature, 
whereby at least a portion of the recorded information corresponds to a feature portion 
of the presentation material as taught by Hull. One of ordinary skill in the art would have 
been motivated to modify this combination because Hull and Chui are from the same 
field of endeavor of electronic presentation material associated with the scan document, 
and provides a predictable result of generating a composite electronic presentation with 
a browser interface for accessing a segment of video file which references scanned 
document- see Chui at Fig. 3 and Col. 4, Line 1-15.) 

In addition Chui and Hull do not teach, but Bozdagi teaches: 

generating a user selectable object providing a user with access to 
the portion of the recorded information corresponding to the visual feature, 
and inserting the user selectable object into the electronic representation 
of the document when the computer system locates a portion of the 
recorded information corresponding to the visual feature, 
(See Fig. 3-4 and the Abstract and Column 5, Line 25 through Column 6, Line 
65^ Bozdagi discloses this limitation in that the storyboarding in real-time and near real- 
time includes the graphical user interface allows a user to visually interact with an input 
video signal to determine the key or representative frames, or to retrieve video 
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segments associated with already determined key frames, wherein the images are 
stored in the memory 70, a compilation of the representative images, can be generated. 
Specifically, the web document 210 shown in FIG. 3 includes a series of representative 
images 115, and their respective accompanying text 117 (i.e. items 115 are the 
selectable objects, since they are linkable to the video files). 
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FIG. 3 

Accordingly, It would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify Chui and Hull method of linking a video to a scan 
document and comparing the visual feature to the recorded information to determine a 
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portion of the recorded information corresponding to the visual feature, whereby at least 
a portion of the recorded information corresponds to a feature portion of the 
presentation material to include a means of generating a user selectable object 
providing a user with access to the portion of the recorded information corresponding to 
the visual feature, and inserting the user selectable object into the electronic 
representation of the document when the computer system locates a portion of the 
recorded information corresponding to the visual feature as taught by Bozdagi. One of 
ordinary skill in the art would have been motivated to modify this combination because 
Chui, Hull and Bozdagi are from the same field of endeavor of electronic presentation 
material (i.e. multimedia) associated with the scan document, and provides a 
predictable result of generating a composite electronic presentation with a browser 
interface for accessing a segment of video file which references scanned document- 
see Chui at Fig. 3 and Col. 4, Line 1-15). 

Regarding Independent claim 21, 

is fully incorporated similar subject of claim 1 cited above, and is similarly 
rejected along the same rationale. Thus, Chui, Hull and Bozdagi disclose every 
limitation of Claim 21 and provide proper reasons to combine, as indicated in the 
above rejections for Claim 1 . 
In addition, Chui teaches: 
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generating composite information based on the portion of in the 
recorded information that corresponding to the visual feature and the 
electronic representation of the document. 

(See Chui at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. annotations 
made on the paper documents during the meeting can be extracted and used as 
indexes to the video. Chui further discloses a method for linking a scanned document 
to a segment of a video is provided. Whereby the scanned document identifier is then 
compared to the plurality of video frame identifiers; the scanned document is linked to a 
first video frame in the plurality of video frames responsive to the comparison step-See 
Chui at Col. 2, Lines 15-25, discloses. 

Also see Chui at Fig. 2, illustrates bar 104a represents the time and duration in 
which document 101 is presented as projection 101b in window 203.) 

Regarding Independent claim 34, 

Claim 34 recites a computer program product stored on a computer- 
readable medium for implement a method recited in Claim 1 . Thus, Chui, Hull 
and Bozdagi disclose every limitation of Claim 34 and provide proper reason to 
combine, as indicated in the above rejections for Claim 1- Also See Chui at 
Fig. 1. 
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Regarding Independent claim 44, 

Claim 44 recites a computer program product stored on a computer- 
readable medium for implement a method recited in Claim 21. Thus, Chui, Hull 
and Bozdagi disclose every limitation of Claim 44 and provide proper reason to 
combine, as indicated in the above rejections for Claim 21- Also See Chui at 
Fig. 1. 

Regarding Independent claim 51, 

Claim 51 recites a data processing system for implement a method recited 
in Claim 1 . Thus, Chui, Hull and Bozdagi disclose every limitation of Claim 51 
and provide proper reason to combine, as indicated in the above rejections for 
Claim 1 - Also See Chui at Fig. 1 . 

Regarding Independent claim 61, 

Claim 61 recites a data processing system for implement a method recited 
in Claim 21 . Thus, Chui, Hull and Bozdagi disclose every limitation of Claim 61 
and provide proper reason to combine, as indicated in the above rejections for 
Claim 61 - Also See Chui at Fig. 1 . 



Regarding Independent claim 68, 

Claim 68 recites a system for implement a method recited in Claim 1 . 
Thus, Chui, Hull and Bozdagi disclose every limitation of Claim 68 and provide 
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proper reason to combine, as indicated in the above rejections for Claim 1- Also 
See Chui at Fig. 1. 

Regarding Independent claim 69, 

Claim 69 recites a data processing system for implement a method recited 
in Claim 21 . Thus, Chui, Hull and Bozdagi disclose every limitation of Claim 69 
and provide proper reason to combine, as indicated in the above rejections for 
Claim 1 - Also See Chui at Fig. 1 . 

Regarding claims 2, 22, 35, 45, 52, and 62, 
Chui further comprises: 

determining associating information for the recorded information 
that corresponds to the extracted feature. 

(See Chui at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. annotations 
made on the paper documents during the meeting can be extracted and used as 
indexes to the video. Chui further discloses a method for linking a scanned document 
to a segment of a video is provided. Whereby the scanned document identifier is then 
compared to the plurality of video frame identifiers; the scanned document is linked to a 
first video frame in the plurality of video frames responsive to the comparison step-See 
Chui at Col. 2, Lines 15-25, discloses. 
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Also see Chui at Fig. 2, illustrates bar 104a represents the time and duration in 
which document 101 is presented as projection 101b in window 203.) 

Regarding claims 3, 23, 46 and 63, 
Chui further comprises: 

wherein the association information comprises time information and 
source information for recorded information. 

(See Chui at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. annotations 
made on the paper documents during the meeting can be extracted and used as 
indexes to the video. Chui further discloses a method for linking a scanned document 
to a segment of a video is provided. Whereby the scanned document identifier is then 
compared to the plurality of video frame identifiers; the scanned document is linked to a 
first video frame in the plurality of video frames responsive to the comparison step-See 
Chui at Col. 2, Lines 15-25, discloses. 

Also see Chui at Fig. 2, illustrates bar 104a represents the time and duration in 
which document 101 is presented as projection 101b in window 203.) 

Regarding claims 4, 36, and 53, 

Chui further comprises: 

associating the association information with the determined additional 
information in the composite electronic representation. 
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(See Chui at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. annotations 
(i.e. additional information) made on the paper documents during the meeting can be 
extracted and used as indexes to the video.) 

Regarding claims 5, 24, 37,47, 54 and 64, 

Chui further comprises: 

receiving a selection to the determined additional information in the 

composite electronic representation and using the association information 

for the additional information to access the recorded information. 
(See Chui at Col. 2, Lines 15-25, discloses a method for linking a scanned document to 
a segment of a video is provided. Whereby the scanned document identifier is then 
compared to the plurality of video frame identifiers; the scanned document is linked to a 
first video frame in the plurality of video frames responsive to the comparison step. 

Also see Chui at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 300 for 
accessing a segment of video file 200 which references scanned document 101a.) 

Regarding claims 6, 25, 38 and 55, 
Chui further comprise: 

accessing the recorded information using the determined additional 
information. 
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(See Chui at Col. 2, Lines 15-25, discloses a method for linking a scanned document to 
a segment of a video is provided. Whereby the scanned document identifier is then 
compared to the plurality of video frame identifiers; the scanned document is linked to a 
first video frame in the plurality of video frames responsive to the comparison step. 

Also see Chui at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 300 for 
accessing a segment of video file 200 which references scanned document 101a.) 

Regarding claims 7, and 26, 
Chui further comprises: 

displaying the accessed recorded information. 

(See Chui at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 300 for accessing 
a segment of video file 200 which references scanned document 101a.) 

Regarding claims 8, and 27, 
Chui further comprises: 

playing the accessed information. 

(See Chui at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 300 for accessing 
a segment of video file 200 which references scanned document 101a.) 



Regarding claims 9, 29, 39, 49, 56 and 66, 

Chui further comprises: 
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performing at least one of emailing, printing, storing, and copying 
the created composite electronic representation. 

(See Chui at Fig. 5 and Col. 8 Line 5, discloses memory item 806 and persistent 
storage 808 stores a scanned digital document 101a and video file 200.) 

Regarding claims 10, 28, 40, 48, 57 and 65, 

Chui further comprises: 

determining metadata using the recorded information that 

corresponds to the feature, wherein the composite electronic 

representation includes the metadata. 
(See Chui at Col. 2, Lines 15-25, discloses a method for linking a scanned document to 
a segment of a video is provided. Whereby the scanned document identifier is then 
compared to the plurality of video frame identifiers; the scanned document is linked to a 
first video frame in the plurality of video frames responsive to the comparison step. 

Also see Chui at Fig. 3 and Col. 4, Line 1-15, discloses browser interface 300 for 
accessing a segment of video file 200 which references scanned document 101 (i.e. 
metadata).) 
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Regarding claims 11, 41 and 58, 
Chui further comprises: 

wherein the received electronic representation of the paper 
document includes notes taken by a user, wherein the created composite 
electronic representation includes the notes taken by the user. 
(See Chui at Col. 7, Lines 1-10, discloses a method of extract the ink annotations, a 
simple comparison between the original and the annotated paper handout may be 
performed. When more than one set of handouts have been annotated, these ink 
strokes may be extracted and selectively layered over the common background of the 
scanned document. Another way to display the ink annotations and notes is simply to 
show them without a background. In any case, the ink strokes may be hyperlinked to 
play their corresponding segment in the video recording.) 

Regarding claim 13, 

Chui further comprises: 

determining a document that includes the recorded information 
using the extracted feature. 

(See Chui at Col. 7, Lines 1-10, discloses a method of extract the ink annotations, a 
simple comparison between the original and the annotated paper handout may be 
performed. When more than one set of handouts have been annotated, these ink 
strokes may be extracted and selectively layered over the common background of the 
scanned document. Another way to display the ink annotations and notes is simply to 
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show them without a background. In any case, the ink strokes may be hyperlinked to 
play their corresponding segment in the video recording.) 

Regarding claim 14, 

Hull further comprises: 

determining a portion of the document that includes the information 
corresponding to the feature. 

(See Fig. 2-33 and Section 4.1 "Similarity detection" Pages 39-40^ Hull discloses this 
limitation in that features extracted from the CCITT fax-compressed representations 
Of two images are compared to determine their visual similarity and whether they are 
equivalent.) 

Accordingly, It would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify Chui and Bozdagi to include a means of determining a 
portion of the document that includes the information corresponding to the feature as 
taught by Hull. One of ordinary skill in the art would have been motivated to modify this 
combination because Hull and Chui are from the same field of endeavor of electronic 
presentation material associated with the scan document, and provides a predictable 
result of generating a composite electronic presentation with a browser interface for 
accessing a segment of video file which references scanned document- see Chui at Fig. 
3 and Col. 4, Line 1-15.) 



Regarding claims 17-18, 
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Chui further comprises: 

receiving the electronic representation comprises receiving a scan of 

the document, the document being a paper document, wherein receiving 

the electronic representation comprises determining an electronic image of 

the document, the document being a paper document. 
(See Chui at Col. 7, Lines 1-10, discloses a method of extract the ink annotations, a 
simple comparison between the original and the annotated paper handout may be 
performed.) 

Regarding claims 19, and 33, 
Chui further comprises: 

wherein receiving the electronic representation comprises receiving 
the electronic representation in response to an input from a user indicating 
that the composite electronic representation should be created. 
(See Chui at the Abstract, teaches a computer system for creating a composite 
electronic representation including presentation material information (i.e. annotations 
made on the paper documents during the meeting can be extracted and used as 
indexes to the video. This interpretation is supported by Applicant's Specification, which 
states "creating a composite electronic representation comprising the determined 
information" at Pages 3 Para 15. ) 
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Regarding claims 20, 30, 43, 60 and 67, 
Chui further comprises: 

the document comprises a paper document. 

(See Chui at Col. 7, Lines 1-10, discloses a method of extract the ink annotations, a 
simple comparison between the original and the annotated paper handout may be 
performed.) 

Claims 15-16, 31-32, 42, 50 and 59 are rejected under 35 U.S.C. 103(a) as 
being as being unpatentable over Chiu et al US007051271B1 filed 10/03/2000 
(hereinafter Chiu), in view of Hull [Document image similarity and equivalence 
protection] Published 11/29/119 by " IJDARUS" Pages 37-42 (hereinafter Hull), and 
further view of Bozdaqi et al -US006647535B1 1 filed 08/1 8/1 999 (hereinafter Bozdagi), 
and further in view of Coar US 200701 06932A1 Continuation of 09/533,152 filed 
03/23/2000 (hereinafter Coar). 

Regarding claims 15, 31, 42, 50 and 59, 

Chui, Hull and Bozdagi do not expressly teach, but Coar teaches: 

an identifier to a location in the recorded information, wherein the 

information in the recorded information corresponding to the feature is 

determined using the identifier. 
(See Fig. 1 and at Para 86-> Coar discloses this limitation in that a user with access to 
the portion of the recorded information corresponding to the visual feature (i.e. XWPL 
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(extensible Workflow Package Language is an extensible Markup Language (XML) 
based language. One portion of the language defines a standardized method to place 
information into a machine-readable symbol such as a high-density barcode) symbols 
(one symbol for each document and row in the data source) for each document to be 
scanned. The user would then prepare the documents with the XWPL symbol in the 
proper sequence to permit scanning, the user could then scan the documents, and the 
application would then, using the necessary symbol recognition tools, extract the data 
from the symbol, and automatically place the image of the document within the correct 
VirPack in the correct placemen.) 

Accordingly, It would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify Chui, Hull and Bozdagi to include a means of said an 
identifier to a location in the recorded information, wherein the information in the 
recorded information corresponding to the feature is determined using the identifier as 
taught by Coar. One of ordinary skill in the art would have been motivated to modify this 
combination because Chui, Hull, Bozdagi and Coar Hull are from the same field of 
endeavor of electronic presentation material associated with the scan document, and 
provides a predictable result of generating a composite electronic presentation with a 
browser interface for accessing a segment of video file which references scanned 
document- see Chui at Fig. 3 and Col. 4, Line 1-15.) 
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Regarding claims 16 and 32, 

Chui, Hull and Bozdagi do not expressly teach, but Coar teaches: 

the identifier comprises at least one of a barcode and signature 
information. 

(See Coar at Fig. 1 and at Para 86, discloses a machine-readable symbol such as a 
high-density barcode.) 

Accordingly, It would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify Chui, Hull and Bozdagi to include a means of said the 
identifier comprises at least one of a barcode and signature information as taught by 
Coar. One of ordinary skill in the art would have been motivated to modify this 
combination because Chui, Hull, Bozdagi and Coar Hull are from the same field of 
endeavor of electronic presentation material associated with the scan document, and 
provides a predictable result of generating a composite electronic presentation with a 
browser interface for accessing a segment of video file which references scanned 
document- see Chui at Fig. 3 and Col. 4, Line 1-15.) 

It is noted that any citations to specific, pages, columns, lines, or figures in the 
prior art references and any interpretation of the references should not be considered to 
be limiting in any way. A reference is relevant for all it contains and may be relied upon 
for all that it would have reasonably suggested to one having ordinary skill in the art. 
See, MPEP2123. 
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Response to Argument 

Applicant's Remarks filed 04/1 1/2008 have been fully considered but they are 
moot in view of the new ground(s) of rejection. 

This office action is a Non-Final Rejection in order to give the applicant sufficient 
opportunity to response to the new line of rejection. 

It is noted, the examiner maintains Chui and Coar references and introduces Hull 
and Bozdagi at this time; since Chui at Fig.1 and 5 discloses browser interface (item 
300) for accessing a segment of video file (item 200) which references scanned 
document (item 101a), wherein browser interface 300 includes window (item 302) for 
viewing scanned documents (item 301 and 1 01 a). This allows a user may double-click 
on the scanned document (item 301 or 101a) and browser interface (item 100a) will play 
in window (item 203) the corresponding recorded segment of event (item 50) 
referencing the scanned documents. Also the scanned documents (item 301 and 101a) 
may be represented by hyperlinked universal resource location ("URL") addresses, 
where the video may be resided- See Chiu at Fig. 1-5 and at Column 3, Line 45 through 
Column 4, Line 15. In addition, Coar discloses a machine readable symbol at the time 
which the paper document printed, wherein those machine readable symbol are 
programmatic interface to permit interaction with all portions of the package (i.e. the 
resulting electronic package file would then contain an image of the paper document 
and the index data in a form that permits easy extraction and transfer to another 
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system- See Coar at Para 31), permit security access, edit, view, extraction, processing, 
and delivery of contents- See Coar at Para 23, 31 , and 41-42.) 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Quoc A. Tran whose telephone number is 571-272- 
8664. The examiner can normally be reached on 9AM - 5PM EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Doug Hutton can be reached on 571-272-4137. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

IQuocA, Ttqnl 
Patent Examiner 

I (Doug Mutton/ 
Doug Hutton 
Supervisory Primary Examiner 
Technology Center 2100 



